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Abstract: Analyzing a conference, especially one as young and focused as LAK, provides the 
opportunity to observe the structure and contributions of the scientific community around it. 
This work will perform a Scientometric analysis, coupled with a more in-depth manual content 
analysis, to extract this insight from the proceedings and program of LAK 2013. Authorship 
analysis reveals an open and international community, while internal citation analysis provides 
evidence of the construction of a body of knowledge central to Learning Analytics. The analysis of 
the content of the papers found five main topics: visualization, behaviour analysis, social learning 
analytics, learning analytics for MOOCs, and learning analytics issues (ethical, scalability, etc.), as 
well as papers reflecting on the field itself. We discuss representative papers presented at the 
conference, highlighting trends and new developments. Learning analytics is a diverse 
multidisciplinary field with an emerging interdisciplinary core, well situated to benefit from 
productive dialogue concerning its scope and purpose and to reflect on the pedagogies and 
epistemologies implied by its methods. 
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1 INTRODUCTION 

Learning Analytics is a new, expanding field that grows at the confluence of learning technologies, 
educational research, and data science. From early works, such as Tinto's (1993) study on factors 
affecting the persistence of college students, to field-defining expositions, such as Siemens and Long 
(2011) and Campbell et al.'s (2007) work on Academic Analytics, to the current papers in this special 
issue, researchers and practitioners with different backgrounds and methodologies have tried to solve 
two simple but challenging questions: How do we measure the important characteristics of the learning 
process? And how do we use those measurements to improve it? 

To assemble this disparate group of researchers and practitioners, the first Learning Analytics and 
Knowledge (LAK) conference was organized in 2011 in Banff, Alberta, Canada. This first event, with only 
27 presentations and 34 different authors, served as the nucleus for what is now the Learning Analytics 
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community. The success of this event led to the organization of LAK 2012 in Vancouver, British 
Columbia, Canada, this time with 52 presentations and 140 authors. LAK 2012 recognized the existence 
of the community and set the basis for an annual conference in the field. The articles presented in this 
special issue belong to the third iteration of this community meeting, LAK 2013, organized in Leuven, 
Belgium. The focus of this third year was Learning Analytics as a "middle-space" where different 
disciplines could exchange visions, ideas, and methodologies to improve learning through a deep 
understanding of the processes in which students, instructors, and institutions are involved. 

As a transition point from a fledgling to a consolidated community, LAK 2013 provides a snapshot of the 
history of Learning Analytics. The goal of this issue of the journal is to provide an overview of the 
composition and content of the LAK 2013 program. This overview will be conducted in two ways: 1) a 
Scientometric analysis of the articles published in LAK '13, and 2) a review of the content of 
representative conference papers. Our conclusions will highlight the main findings of these two 
approaches. 

2 SCIENTOMETRIC ANALYSIS 

All published scientific endeavours leave traces susceptible to quantitative analysis in order to gain a 
better understanding of epistemological phenomena occurring in the scientific field. Co-authorship and 
bibliographical coupling networks could provide insight into the nature of the scientific community and 
how the individuals in it influence each other (Kessler, 1963). Text processing of the content of the 
scientific papers could provide evidence for the emergence or disappearance of research topics and 
concepts from the research field (Ding, Chowdhury, & Foo, 2001). Simple statistics about the authors 
and their papers can provide an indication of the reach, diversity, and "health" of the community of 
scientists and practitioners interested in the field. This type of analysis is commonly known as 
Bibliometrics (when related only to the documents produced by research) or more generally as 
Scientometrics (when the focus is the research community and the knowledge building process) (Hood 
& Wilson, 2001). In this section, we study the LAK 2013 program through the Scientometric lens. The 
rest of this section will provide a quantitative analysis of the status of Learning Analytics research as 
presented at LAK 2013. 

2.1 Authorship Analysis 

The first step in understanding the authorship of LAK 2013 is to obtain simple statistics regarding the 
author distribution of the papers presented at LAK 2013: 

Basic Quantities: LAK 2013 had an approval rate of 28% for full papers (58 submitted, 16 accepted) and 
an approval rate of 22% for short papers (36 submitted, 8 accepted) without counting 14 full papers 
accepted as short. By comparison, the acceptance rate for full papers for LAK 2011 and LAK 2012 was 
45% and 39% respectively. The LAK 2013 proceedings contained 47 documents (16 full papers, 22 short 
papers, 2 design briefs, 4 workshops, 2 panels, and 1 note from the editors). A total of 143 authors 
produced those documents. 

New and Returning Authors: At LAK 2011, there were 34 authors. Of those, 30 returned to LAK 2012, 
together with 110 new authors, expanding the community to 140 individuals. At LAK 2013, there were 41 
returning authors from LAK 2011 (4), 2012 (17), or both (20). At LAK 2013, the community annexed 102 
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new authors. 


Authorship distribution: How prolific an author is can be measured by how many papers he or she has 
been involved in. In the case of LAK '13, the most prolific authors are Simon Buckingham Shum (5 
documents), Peter Reimann, Rebecca Ferguson, and Ryan Baker (3 documents each). If their 
contributions are weighted (the weight of a paper is the inverse of the number of authors), the list 
changes to Paulo Blikstein (1.5 papers), Simon Buckingham Shum (1.075 papers), Andreas Harrer, Doug 
Clow, and June Ahn (1 paper each). The distribution is very flat, with no author actually dominating the 
scene. This evidence, together with the fact that most of the authors were new to LAK, suggests that 
Learning Analytics is still an open community where interested researchers can be included without 
being part of a closed "academic club." 



Geographical Distribution: To calculate the geographical distribution, two measurements were 
considered: contribution by author and contribution by document. The first was calculated simply by the 
number of different authors affiliated with each country. The second was calculated by the weighted 
contribution of each author (each paper weight as the inverse of the number of authors). The top 
countries for the first measurement were USA (51 authors), UK (20 authors), Germany (18 authors), and 
Greece and Netherlands (9 authors each). For the second metric, the top countries do not vary much: 
USA (16.65 papers), UK (7.275 papers), Germany (5.9 papers), Greece (3.05 papers) and Canada (2.2 
papers). A continental view shows the following results: Europe contributed with 76 authors, North 
America with 58, Oceania with 6 and Asia, South America, and Africa with 1 author each. A full graph of 
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the distribution (Figure 1) presents a conference still dominated by the main research players 
worldwide: USA, West and Central Europe, Canada, and Australia. While small contributions are present 
from Latin America and Africa, there is a notable absence of Asian contributors. 

Career Length Distribution: The length of the research career of each author was estimated using the 
date of first publication as recorded by the ACM Digital Library. While this data under-represents the 
career length of researchers in the field of education, the values obtained offer a general indication of 
distribution of seniority. For the 143 authors, the distribution was as follows: 28 senior researchers 
(>=15 years), 45 junior researchers (<=14 years and >4 years) and 70 Ph.D. students or initial stage 
researchers (<=4 years). This distribution reflects the pyramidal structure of most scientific groups (see 
Figure 2). 


70 



Figure 2. Distribution of Career Length 


Sector Distribution: Authors were classified in three sectors (Academia, Research, and Industry) 
according to their affiliation. The results show that 128 authors belong to academic institutions, 8 to 
research centres, and 7 to companies. LAK '13 can be clearly classified as a research-oriented 
conference. 

Additional information about the Learning Analytics community can be extracted from how different 
individuals collaborate in order to produce research. The following is a series of collaboration indicators: 

Authors per Paper: The average is 3.44 authors, which means that LAK 2013 publications usually needed 
a team of at least three people. This result maintains when the distribution of authors per paper is 
analyzed (Figure 3). Teams of two, three, four, and five individuals produced most of the papers rather 
than single authors. 



Figure 3. Distribution of Authors per Paper 
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Institutional Collaboration: We consider a paper inter-institutional if its authors are affiliated with more 
than one institution from the same country. We consider a paper international if its authors are 
affiliated with at least two institutions in different countries. Analysis of the collaboration shows that, of 
the 47 documents, 14 (30%) were international, 8 (17%) were inter-institutional, and 25 (53%) were 
produced by authors of a single institution. These values reflect the current percentage of collaboration 
in science (Leydesdorff & Wagner, 2008). 
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Figure 4. Co-Authorship Network: Size and colour represent the number of published 

papers from LAK 2013 

Co-authorship Network: A useful way to visualize the author community is to draw a co-authorship 
network. In this network, each node is an LAK '13 author and each line is a link between co-authors. The 
strength of the line depends on the number of LAK '13 papers that the two authors have co-authored 
(see Figure 4). Most authors, the authors of individual papers, congregate in small cliques. It is 
interesting to note, however, that a more intricate collaboration network interconnects several 
important LAK '13 authors. Simon Buckingham Shum, Rebecca Ferguson, Roy Pea, Maarten de Laat, and 

ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) n 







JOURNAL OF LEARNING ANALYTICS 


S-LAR 


(2014). Analysis and Reflections on the Third Learning Analytics and Knowledge Conference (LAK 2013). Journal of Learning Analytics, 1(2), 5-22. 


Ryan Baker are part of this sub-network. Other small nuclei of collaboration surround Stephanie Teasley 
and Peter Reimann. A parallel analysis of all Learning Analytics literature conducted by Dawson, Gasevic, 
Siemens, and Joksimovic (2014) found a similar network configuration. 

2.2 Internal Citation Analysis 

Scientific papers are not isolated entities; each new paper is linked to previous papers through citations. 
It is possible to gain insight on the main sources of knowledge in a field through the analysis of which 
papers and authors are the most referenced and how different papers may be similar based on their 
number of references in common. The following provides a basic quantitative analysis for the references 
contained in the LAK '13 papers: 

Basic Quantities: LAK 2013 documents had a total of 929 references, with an average of 21 bibliographic 
entries per paper, suggesting that the LAK papers draw knowledge from a large number of sources. 

Shared References: The references present in LAK 2013 documents were directed to 819 different 
papers. Only 8% of the referenced papers (68 papers) are shared between at least two publications. 

Most Referenced Papers: The most referenced articles of LAK 2013 are presented in Table 1. As can be 
seen, all of the top-12 papers are specifically Learning Analytics papers, published in previous editions of 
LAK or in related Learning Technology venues that existed before LAK. This result complements the one 
found in the previous analysis. Even though small, there is a body of knowledge central to Learning 
Analytics and new papers in the field use it. The maturity of the field can be judged again by the nature 
of the most referenced papers. Most are definitions of the concept of Learning Analytics or early 
examples of its use. This focus of citation on conceptual papers was also found in Dawson et al. (2014). 
The number of references to article follows a traditional long-tailed distribution. 

Most Referenced Authors: The references found in LAK 2013 papers also allow us to find the more 
relevant scientists in the field, at least from the perspective of this particular conference. The top nine 
authors can be seen in Table 2. The results, not surprisingly, relate to the general perception of leaders 
in the field. The distribution of references to authors also follows the long-tailed distribution common to 
scientific citation. 

Bibliographic Coupling Network: Two papers are bibliographically coupled if they both reference the 
same third paper. The strength of the coupling depends on the number of papers referenced in 
common. Figure 5 shows a network where each node is a paper published in LAK 2013 and the edges 
between them represent the strength of their bibliographical coupling. The size of each node represents 
the number of other papers to which that node is coupled. As Figure 5 shows, most of the papers are 
coupled to at least one other paper. While still nascent, the strength of the core network (papers 3, 26, 
28, 31, 32, 33) 1 seems to validate a common body of knowledge that Learning Analytics considers its 
own. These papers seem to lay out the bounds of the field. Of the ten disconnected nodes, five are 
papers (7 is a long paper and 15, 30, 38, and 41 are short papers) and five are workshops and panels, 
which is not surprising since such documents contain few or no references. 


1 Paper 3: Santos, Verbert, Govaerts, and Duval (2013); paper 26: Clow (2013); paper 28: Dimopoulos, Petropoulou, and Retails 
(2013); paper 31: Monroy, Snodgrass Rangel, and Whitaker (2013); paper 32: Dyckhoff, Lukarov, Muslim, Chatti, and 
Schroeder (2013); paper 33: Camilleri, de Freitas, Montebello, and McDonagh-Smith (2013). 
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Table 1. References most cited by LAK 2013 papers 


Paper Title 

No. of 
References 

Ferguson, R., & Buckingham Shum, S. (2012, April). Social learning analytics: Five 
approaches. Proceedings of the Second International Conference on Learning 
Analytics and Knowledge (pp. 23-33). ACM. 

7 

Ferguson, R. (2012, March). The state of learning analytics in 2012: A review and 
future challenges. Knowledge Media Institute. 

6 

Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning 
through educational data mining and learning analytics: An issue brief. US 
Department of Education, Office of Educational Technology, 1-57. 

6 

Buckingham Shum, S., & Ferguson, R. (2012). Social Learning Analytics. Educational 
Technology & Society, 15(3), 3-26. 

5 

Arnold, K. E., & Pistilli, M. D. (2012, April). Course Signals at Purdue: Using learning 
analytics to increase student success. Proceedings of the Second International 
Conference on Learning Analytics and Knowledge (pp. 267-270). 

5 

Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012, May). The student activity 
meter for awareness and self-reflection. CHI '12: Extended Abstracts on Human 
Factors in Computing Systems (pp. 869-884). ACM. 

5 

Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 
33(1), nl. 

5 

Clow, D. (2012, April). The learning analytics cycle: Closing the loop effectively. 
Proceedings of the Second International Conference on Learning Analytics and 
Knowledge (pp. 134-138). ACM. 

4 

Siemens, G. (2012, April). Learning analytics: Envisioning a research discipline and a 
domain of practice. Proceedings of the Second International Conference on Learning 
Analytics and Knowledge (pp. 4-8). ACM. 

4 

Oblinger, D., & Campbell, J. (2007). Academic analytics, Educause white paper. 
Retrieved 20 October 2011. 

4 

Elias, T. (2011). Learning analytics: Definitions, processes and potential. Learning, 
23, 134-148. 

4 

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and 
education. Educause Review , 46(5), 30-32. 

4 


Table 2. Top Nine Referenced Authors in LAK 2013 


Author 

Times Referenced 

No. Papers in which Referenced 

G. Siemens 

22 

16 

R. Ferguson 

27 

15 

E. Duval 

27 

13 

S. Buckingham Shum 

20 

11 

K. Verbert 

22 

10 

R. S. J. d. Baker 

21 

8 

S. Dawson 

14 

8 

A. Pardo 

ii 

8 

K. Koedinger 

11 

8 
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Figure 5. Bibliographic Coupling Network: Each node represents a paper published in LAK 2013 
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Figure 6. Paper downloads from ACM Digital Library 
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2.3 External Citation Analysis 


While one year on is still too early to determine the relevance of the papers published in LAK 2013 by 
the number of citations, the number of times they have been accessed or downloaded from the ACM 
Digital Library could be used as a proxy indication of the impact of the different papers in the field (see 
Figure 6). The most downloaded paper — "MOOCs and the funnel of participation" (Clow, 2013) — had 
been downloaded 732 times by the time of this study. The second most downloaded paper — 
"Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses" 
(Kizilcec, Piech, & Schneider, 2013) — has been downloaded 444 times. These two papers deal with a 
trending topic: Massive online open courses (MOOCs). This result suggests that the early visibility of the 
papers and by proxy, their early impact on the field, is strongly connected to the popularity of the topics 
they discuss. 


2.4 Word Analysis 

Counting the number of times a word appears in a set of documents usually correlates to the 
importance of the underlying concept to the topic. The Carrot 2 clustering engine (Stefanowski & Weiss, 
2003) was used to detect the main words used in LAK 2013 papers and to find related words that appear 
frequently with those important words. This analysis provides an idea of the main concepts discussed in 
Learning Analytics papers. Figure 7 presents the seven most used, non-stop words in LAK 2013 
documents. The size of each word relates to its relative frequency (number of documents in which the 
word appears divided by the number of documents) in the document set. Seven words have a relative 
frequency of larger than 0.1 (10%): students, data, analytics, learning, use, activity, and education. These 
words can be considered the ad-hoc definition of Learning Analytics: use of student and activity data to 
improve learning and educational processes. A more detailed view of the word analysis can be seen in 
Figure 8, which shows the most common words that appear together with the most frequent words. 
Here concepts used in Learning Analytics can be found: student performance, student interaction, 
datasets, linked data, process analytics, learning environments, learning process, collaborative learning, 
learning activities, activity theory, activity streams, higher education, educational research, etc. 
Comparing these words to similar analyses for LAK 2011 and LAK 2012 shows some consolidation in 
concepts (a lower number of frequent terms) and an increase in the richness of concepts (more related 
words for each frequent word). In summary, the word analysis suggests a conference with increased 
focus and depth. 

3 CONTENT ANALYSIS 


3.1 Identified Topics 

The LAK '13 program grouped papers into 14 sessions (Suthers & Verbert, 2013): reflections on learning 
analytics, visualization for reflection and awareness, social learning analysis and visualization, 
communication and collaboration, discourse analytics, behaviour analysis, affect analytics, predictive 
analytics, sequence analysis, MOOCs, assessment, supporting teachers, challenges, and design briefings. 

If the classification had been done without restrictions regarding timetable or session length, these 
papers could have been clustered into six categories: 
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Figure 8. Words related to most frequent words in LAK 2013 papers 
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1. Visualization. Papers in this category focus on increasing reflection and awareness through 
visualization techniques. Dashboard applications are presented as well as evaluation studies that 
assess the impact of such dashboards on student learning. Visualization methods appear throughout 
the other categories, reflecting their relevance to multiple stakeholders. 


2. Behaviour analysis. The first of four categories of these papers offers advances in making higher- 
level inferences from low-level actions by capturing and analyzing different data sources (gesture 
and object manipulation, eye tracking data, data captured from Kahn academy). A second category 
focuses specifically on effects; for example, to estimate confidence or level of expertise from eye 
tracking data. A third, predictive analytics, tries to predict at-risk students and to show the 
development of new media learning skills from participation in Facebook. A final category focuses 
on analysis of the sequential structure of student actions; for example, to categorize student 
trajectories for exploring a game environment on fractions. 


3. Social learning analytics. This area combines both analysis and visualization techniques to gain 
insight into social behaviour. In the first of three sections, research focuses on analysis and 
visualization of social interactions to make people aware of their social context. In the second 
collection, communication and collaboration is analyzed, focusing on both reading and writing 
activities, including analysis of paragraph level revisions and how collaboration forms around topics 
in collaborative writing. The last section focuses on discourse analytics from diverse perspectives: 
scientometric path analysis (citation analysis that includes temporality) to model knowledge 
evolution; a comparative evaluation classification approach for identifying exploratory dialogue; and 
theoretical reflections on how learning analytics embody assumptions about both our own and 
learners' epistemologies, with applications for the analysis of user trace data. 

4. Applications. Beginning with learning analytics applied to Massive Open Online Courses (MOOCs) in 
which large numbers of people participate with an open enrollment model, applications of learning 
analytics form an important topic. Assessment and teacher support — such as by visualizing data 
collected in real applications for monitoring and direct content development — are both important 
objectives of several applications. 


5. Challenges. This group of papers discusses challenges such as scalable learning analytics, ethics and 
institutional policies for using student data, and collection and sharing of datasets for research 
purposes. These challenges include the design and development of architectures for organizing the 
analytic enterprise across micro and macro levels of activity, and across the various tools used in a 
collaborative learning environment. 

6. Definitional. A few papers reflect on learning analytics itself, including its theoretical and 
pedagogical assumptions and examination of the identity and interdisciplinary nature of the field. 

3.2 Representative Papers 

In this section, we highlight representative papers that address the identified topics. The papers chosen 

include those nominated for the best paper award and others representative of their categories. 
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3.2.1 Visualization 

Santos, Verbert, Govaerts, and Duval (2013) present a study that assesses the use of visualizations to 
support learning analytics. The overall objective is to use learning analytics dashboards that incorporate 
relevant visualizations to support awareness, self-reflection, and sense-making for learners. Although 
several learning analytics dashboards have been discussed in recent literature (Verbert et al., 2013), 
little research has been done to evaluate the usefulness of these dashboards for learners. Santos et al. 
(2013) present the results of an evaluation study of the StepUp! dashboard, and the extent to which it 
addresses the issues and needs of learners as collected in brainstorming sessions. The authors then 
evaluate to what extent the dashboard addresses these issues and needs. 


Another example of visualization is the paper by Southavilay, Yacef, Reimann, and Calvo (2013), who 
visualize document revisions, topic evolution, and collaboration networks based on feedback concerning 
the individual and collective agency of collaborative writers and their teachers, which leads to the next 
category of behaviour analysis. 

3.2.2 Behaviour Analysis 

Some papers in this category utilize sequential analysis at granularities ranging from the individual 
actions of learners to community learning processes over time. For example, main path analysis is a 
method from sociometrics for identifying key publications in the development of a scientific field by 
tracing paths in the directed acyclic graph of citations. Halatchliyski, Hecking, Gohnert, and Hoppe 
(2013), a best paper nominee, apply main path analysis to the evolution of artifacts in a collaboratively 
edited Wikiversity to analyze the introduction and evolution of ideas in the community. A flexible 
analytic workbench supports the work. At the other extreme of granularity, which the authors call 
"nanogenetic analysis/' Martin et al. (2013) use keystroke-level data to track and classify student 
trajectories in the state-space of a game. 

Two salient and well-received papers in this area explore alternative data sources for learning analytics 
based on embodied activity. The paper by Schneider, Abu-EI-Haija, Reesman, and Pea (2013), a best 
paper nominee, also exemplifies sequential analysis at the level of individual actions. Schneider et al. 
used eye tracking on collaborating dyads and represented gaze location and transitions as graphs. They 
combined the graphs for each dyad to represent shared gaze. Graphs with denser shared gaze locations 
and transitions indicate mutual orientation towards the task. Further research is proposed to identify 
network metrics that can predict task performance. Worsley and Blikstein (2013) present research on 
how to identify expertise of students automatically from object manipulation data and gestures. More 
specifically, multimodal learning analytics techniques for understanding and identifying expertise as 
students engage in a hands-on building activity are presented. The authors were able to identify key 
elements in how to segment and compress object manipulation codes. In addition, they show how 
dynamic time warping combined with clustering can be used to classify student expertise accurately. 
The research outcomes also motivate further promising, but challenging, research in the area of 
behaviour analysis with a wide variety of sensing devices. 

Alternative metrics can also be considered within online environments. Wise, Zhao, and Hausknecht 
(2013) draw our attention to the importance of "invisible behaviour/' such as student message reading, 
as well as message writing in online forums. They also discuss a framework for applying learning 
analytics that includes their integration with the goals of learning, student agency in interpreting diverse 
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measures, providing time and space for learner reflection on analytics, and supporting discussion 
between students and instructors as partners. 

3.2.3 Social Learning Analytics 

Social learning analytics (Buckingham Shum & Ferguson, 2012) is one of the more active areas of 
research and discussion; the most cited paper at LAK '13 was an overview of the topic (Ferguson & 
Buckingham Shum, 2012). Interesting work on social learning analytics, focusing on understanding the 
literacies embedded in social media participation, was presented by Ahn (2013). The research employed 
an exploratory factor analysis to examine how raw activity data may group into particular participatory 
practices, as well as regression models that explore whether these participatory practices predict new 
media literacy skills. More specifically, data from Facebook was used to examine relationships between 
the behaviours of learners resulting in a promising approach to identify new media literacy skills related 
to 21st century skills. Many papers in behaviour analysis also addressed social learning concerns. 

3.2.4 Applications 

The application of learning analytics in MOOCs is of major interest, and important challenges related to 
both popularity and low completion rates of learners are a central concern in MOOCs. The highly 
downloaded short paper by Doug Clow (2013) quantifies the drop-off of activity in three online learning 
sites. The full paper by Kizilcec, Piech, and Schneider (2013) goes further, presenting a classification 
method that identifies different engagement trajectories in MOOCs based on learner interaction with 
video lectures and assessments: "completing," "auditing," "disengaging" and "sampling." With this 
classifier, the paper posits a promising approach to deconstructing gained disengagement based on 
interaction data with materials commonly used by MOOCs. The paper also draws our attention to the 
fact that learners may be accessing MOOCs with different motivations, and that, for many of them, 
meeting their objectives does not necessarily include completion. At the time of writing, the paper had 
already been cited 68 times by researchers in the field, illustrating the high potential of this work for the 
learning analytics field (source: Google Scholar). 

3.2.5 Challenges 

A challenge articulated by several researchers is the collection of datasets that can be shared for 
research purposes (Drachsler et al., 2010; Verbert, Manouselis, Drachsler, Duval, 2012). The overall 
objective is to collect data captured in real-life settings, from different learning environments, and to 
make such data available for researchers to enable comparison of research results. Niemann, Wolpers, 
Stoitsis, Chinis, & Manouselis (2013) present research regarding interchanging data collected from 
different learning environments. The authors first studied the data types and formats that learning 
environments use to represent and store learner data. In addition, they developed crosswalks between 
different schemas, so that datasets can be combined. The paper presents interesting issues that must be 
addressed before aggregated sets of learner data can actually be used to support learning analytics 
research. 

3.2.6 Reflections 

Two of the LAK '13 best paper nominees offered reflections on the field itself. A thought provoking 
paper by Knight, Buckingham Shum, and Littleton (2013) examined the relationships between pedagogy, 
assessment, and epistemology, challenging us to identify where we stand in our own learning analytic 
practice. The authors outline the analytic implications of several established pedagogic approaches; for 
example, that "transactional approaches may emphasise use of facts; constructivist the broad (and 
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contextual) application of skills; subjectivist the self-efficacy and motivators of students; apprenticeship 
the dynamic practical based learning which may occur through high level membership of communities 
of practice; connectivism the ability of students to build up, link and curate their knowledge 'networks.'" 
Thus, reasoning in converse, learning analytics are not neutral but may imply or encourage certain 
pedagogical views. The authors go further to argue that learning analytics embody epistemological 
assumptions about the nature of knowledge in how they approach the assessment of knowledge. 
Becoming aware of these epistemological assumptions is consequential because learning analytics can 
be used to perpetuate assessment regimes "detrimental to the wider enterprise of education," or 
alternatively assessment regimes that emphasize learner agency and the contextualized nature of 
knowledge. The paper is recommended for those wanting to approach their analytic practice in a 
deliberate and reflective manner. 

Moving now to an even broader perspective, Balacheff and Lund (2013), grapple with the question of 
how multiple disciplines come together in learning analytics, and its relation to other research areas, 
such as educational data mining. In multidisciplinarity, the subject of study is approached from each 
disciplinary perspective without theoretical, conceptual, or methodological integration, while 
interdisciplinary research involves such integration. A bridge between the two may be found in 
strategies of "productive multivocality" (Suthers, Lund, Rose, Teplovs, & Law, 2013), methods of 
bringing together the various "voices" of different theoretical and methodological traditions in dialogues 
that not only provide different points of view on the subject but also expose assumptions and make 
epistemological positions explicit. Balacheff and Lund (2013) call for examining and making explicit the 
"problematiques" of learning analytics: the coherent frameworks by which we express problems and 
why they are interesting to solve along with the approaches for solving them. They challenge us to 
confront "the different understandings of Learning Analytics, without necessarily choosing any one 
particular definition as the one that is destined to become canonical." 

4 CONCLUSIONS 

The main findings of the analyses conducted on LAK '13 can be summarized as follows: 

• LAK '13 was the venue of an open and international community of researchers. This community 
currently consists of a core of 40 researchers that attracts an orbit of 100 additional authors. 

• The statistical distributions of the author community of LAK '13 are in line with that of other 
scientific communities. 

• The body of knowledge central to learning analytics is still small but already shows signs of 
consolidation. 

• Five main topics have been identified in the LAK '13 program — visualization, behaviour analysis, 
social learning analytics, learning analytics for MOOCs, and learning analytics issues (ethical, 
scalability, etc.) — as well as a sixth topic of reflecting on the field itself. 

• Learning analytics is at a crucial stage in its development that calls for individual and collective 
reflection and dialogue to arrive at an understanding of the field. This is not necessarily a single core 
definition, but rather a locus for productive dialogue and collaboration between multiple 
theoretical, methodological, and practical perspectives concerning what problems are important to 
solve and how the analytics we are promoting reflect our epistemological assumptions and 
pedagogical stances. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) 


18 


JOURNAL OF LEARNING ANALYTICS 


S-LAR 


(2014). Analysis and Reflections on the Third Learning Analytics and Knowledge Conference (LAK 2013). Journal of Learning Analytics, 1(2), 5-22. 


This kind of analysis should be performed periodically to gain insight on the evolution of the field, as 
well as to confirm the healthy status of its surrounding community. 
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